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Abstract. Given a finite connected graph G, place a bin at each vertex. Two 
bins are called a pair if they share an edge of G. At discrete times, a ball is 
added to each pair of bins. In a pair of bins, one of the bins gets the ball 
with probability proportional to its current number of balls raised by some 
fixed power a > 0. We characterize the limiting behavior of the proportion 
of balls in the bins when G is regular and a < 1. For example, when G is 
non-bipartite the proportion converges to the uniform measure almost surely. 
A novel part of the proof consists of analyzing the equilibrium set of a non 
uniformly hyperbolic dynamical system. 



1. Introduction and statement of results 

Let G — (V, E) be a finite connected graph with V = {1, . . . , m} and \E\ = N, 
and assume that on each vertex i there is a bin initially with Bi(0) > 1 balls. For a 
fixed parameter a > 0, consider a random process of adding N balls to these bins 
at each step, according to the following evolution law: if the numbers of balls after 
step n — 1 are £?i(n — 1), . . . ,B m (n — 1), step n consists of adding, to each edge 
{i, j} € E, one ball either to i or j, and the probability that the ball is added to i 
is 

P \i is chosen among {i, j\ at step n] = — -. * -. — • (1.1) 

1 &X,J} v 1 Bi(n-l) a +Bj(n-l) a ' 

The number of balls at vertex i after step n is thus Bi(n) = Bi(n — 1) + Cj(n) , 
where Ci(n) is the number of balls added to i at step n. 

In this paper, we study the limiting behavior, as the number of steps grows, of 
the proportion of balls on the vertices of G. More specifically, let N = Y^iLi Bi(0) 
denote the initial total number of balls, let 

Xi(n) = A B ^ U \r , i = l,...,m, (1.2) 
TVo + nis 

be the proportion of balls at vertex i after step n, and let x(n) = (xi(n), . . . , x m {n)). 
Call the point (1/m, . . . , 1/m) the uniform measure. The first result classifies the 
limiting behavior of x(n) when G is regular and a = 1. 

Theorem 1.1. Let G be a finite, regular, connected graph, and let a = 1. 

(a) IfG is non-bipartite, then x(n) converges to the uniform measure almost surely. 

(b) If G is bipartite, then x(n) converges to A almost surely. 
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Above, A is a subset of the (m — l)-dimcnsional open simplex A defined as 
follows: if V — A U B is the bipartition of G, then 

A = {(xi, . . . , x m ) € A : 3p,(; > s.t. X4 = p on A and xi — q on B}. 

Theorem |1.1| includes the case of any finite complete graph. A complete graph 
with at least three vertices is non-bipartite and then the random process converges 
to the uniform measure almost surely. 

Theorem |1.1| also includes the case of cycles: if the length is odd, the random 
process converges almost surely to the uniform measure; if the length is even, the 
random process' limit set is contained in A almost surely. 

The next result shows that, when G is regular and a < 1, x{n) converges to the 
uniform measure, regardless of G being bipartite or not. 

Theorem 1.2. Let G be a finite, regular, connected graph, and let a < 1. Then 
x(n) converges to the uniform measure almost surely. 

The last main result deals with star graphs. The star graph is a tree with m 
vertices and m — 1 leaves. 

Theorem 1.3. Let G be a finite star graph with at least three vertices, and let m 
be the vertex of highest degree. 

(a) If a = 1, then x(n) converges to (0, ... , 0, 1) almost surely. 

(b) If a > 1, then with positive probability x(n) converges to either 

(0,...,0,1) or ( — L^,...,— ^,0 
\ m — 1 m — 1 

Imagine there are 3 companies, denoted by M, A and G. Each company sells two 
products. M sells OS and SE, A sells OS and SP, G sells SE and SP. Each pair 
of companies compete on one product. The companies try to use their global size 
and reputation to boost sales. A natural question is which company will sell more 
products in the long term. In this scenario, one can easily see that the interacting 
relation among the three companies forms a triangular network. On this triangle, 
a vertex represents a company and an edge represents a product. Under further 
simplifications, the model proposed above describes in broad strokes the long term 
evolution of such a competition. 

Another motivation for the model comes from a repeated game in which agents 
improve their skills by gaining experience. The interaction network between agents 
is modeled by a graph. At each round a pair is competing for a ball. A competitor 
improves its skills with time, which is represented by the number of balls in its bin. 

There are several natural ways to generalize our model in order to capture more 
complex interactions, but we focus here on the simplest setup. The model can be 
viewed as a class of graph based evolutionary model, which has been studied in 
various fields, e.g. biology 11 , economics [5], mathematics 15 , and sociology 14 



The classical Polya's urn contains balls of two colors, and at each step one ball 
is drawn randomly, its color is observed, and a ball of the same color is added to 
the urn. In our process the interactions occur among pairs of bins, and on each 
of them the model evolves similar to the classical Polya's urn. We therefore call 
it a generalized Polya's urn with graph based interactions. For other variations on 



Polya's urn, see e.g. |2||8[|10 . 

We use a dynamical approach to prove Theorems 1 1 . 1 1 and 1.2 Benaim [l] proved 



that in many circumstances the limit set of a random process is contained in the 
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chain-recurrent set of an associated semiflow. The novelty of our work is the proof 
of Theorem 1 1 . 1 1 when G is bipartite. The chain-recurrent set of the semiflow is A. 
Unlike in classical situations, A is not hyperbolic: although each point of A is a 
stable singularity, they lose hyperbolicity as they approach the extremes of A, and in 
the limit they degenerate to non-hyperbolic points. To overcome this difficulty, we 
extend the semiflow to neighborhoods of the extremes of A and construct Lyapunov 
functions on these neighborhoods that clarify the local behavior of the extended 
semiflow. 

The paper is organized as follows. In Section [2] we make a brief discussion 
on stochastic approximation algorithms. In Section [3] we introduce the dynamical 
approach. In Sections|4}[5land|6|we prove Theorems |Llj|1.2| and |1.3| respectively. In 
Section [7] we discuss some variants of the model. Finally, we collect some questions 
in Section [3 



2. Stochastic approximation algorithms 

A stochastic approximation algorithm is a discrete time stochastic process whose 
general form can be written as 

x(n + 1) - x{n) = >y n H(x(n),Z(n)) (2.1) 

where H : M. m x W n —> W 11 is a measurable function that characterizes the algo- 
rithm, {x(n)} n > C l m is the sequence of parameters to be recursively updated, 
{£(n)}„> C K m is a sequence of random inputs where H(x(n), £(n)) is observable, 
and {7n},i>o is a sequence of "small" nonnegative scalar gains. Such processes were 
first introduced in the early 50s on the works of Robbins and Monro uM and Kiefer 
and Wolfowitz (6l. 

We now show that the random process defined by (1.2 1 is a stochastic approx- 
imation algorithm. We start by specifically formulating the model. Write i ~ j 
whenever G E. Let T n denote the sigma-algebra of the process up to step n. 

For each neighboring pair of vertices i ~ j, consider 0-1 valued random variables 
5i^j(n + 1), Sj^i(n + 1) such that Si^j(n + 1) + 5j^-i(n + 1) = 1 and 

wtx ( , i\iri Bj{n) a Xj(n) a 

E [S^in + l)\T n ] = Bi{n)a + Bj{n)a - Xi{n)a+Xj(n)a ■ ( 2 - 2 ) 

Also, assume that Si^j(n+1) and ^'^'(n+l) are independent whenever the edges 
and are distinct. Thus 

C i (n + l)=J2 § i^j( n + 1 )- (2-3) 
We want to show that x(n) = (xi(n), . . . ,x m (n)) satisfies a difference equation 

Bi(n)+Ci(n + 1) Bi(n) 



(|2_lj). Observe that 

Xi{n + 1) - Xi(n) 



N + (n+ l)N N + nN 
-Nxi(n) +Ci{n + 1) 



N a + (n + 1)N 



¥T ^ Tiy (-x,,„) + ic,(„ +1) ; 
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and so x(n) satisfies (2.1 1 with 



7n= No ; * ; ^ , £(n) = ~ (C x (n + 1), . . . , C m (n + 1)) (2.4) 
-W + {n + l) TV 

and if : W n x M m W n defined by 

H(x(n),£(n)) = -x(n) + £(ra). 

Conditioning on _F n , ii has a deterministic component x(n) and a random com- 
ponent £(n). Nevertheless, nothing can be said about converging properties of £(n). 
To this matter, we modify the above equation by decoupling into its mean part 
and the so called "noise" part, which has zero mean. If one manages to control the 
total error of the noise term, the limiting behavior of x(n) can be identified via the 
limiting behavior of the new deterministic component. 

3. The dynamical approach 

The dynamical approach is a method introduced by Ljung [9] and Kushner and 
Clark [7] to analyze stochastic approximation algorithms. Formally, it says that 



recursive expressions of the form (2.1 1 can be analyzed via an averaged ordinary 
differential equation (ODE) 

^ =»(*(*)). (3-1) 

where H(x) — Imin^^ E [H(x, £(n))]. 

In this perspective, our stochastic approximation algorithm can be written as 

x(n + 1) - x(n) = 7 „ {(-x(n) + E [£(n)|.F n ]) + (£(n) - E [£(n)\F n ])} . 

Denote £(n) = . . .,f m (n)). Observe that ( pT2] ), pi?} and ([2T) imply 

Thus, defining {u n }„> C W n by 

u n =^(n)-EK(n)|J>,] (3.2) 

and F = (i\, . . . , f 7 ^) to be a vector field in A with 

. . 1 ^ — > Xi{ri) a . k 

Fi[xx, ...,x m ) = -Xi + —y 3.3) 

TV ' Xi(n) a + Xj(n) a 

our random process takes the form 

x(n + 1) - x(n) = 7 „ [F(a(n)) + u n ] . (3.4) 

The above expression is a particular case of a class of stochastic approximation 
algorithms studied by Benai'm in 1 , on which he related the behavior of the algo- 
rithm to a weak notion of recurrence for the ODE: that of chain-recurrence. His 
theorem asserts that, under the assumptions of Kushner and Clark lemma [7], the 
accumulation points of {x(n)} n >o are contained in the chain-recurrent set of the 
semiflow generated by the ODE. 

In the remaining of this section, we introduce the necessary definitions for semi- 
flows, then state Benai'm's Theorem, and conclude the section proving that our 
model satisfies the required conditions of his theorem. 
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3.1. Preliminaries on semiflows. Let U C M m be an open set, and let $ : 
M>o x U — >• U be a continuous map. For simplicity, denote &(t,x) by ^((cc). 

Definition 3.1 (Scmiflow). A semiflow on U is a continuous map $ : M>o x [/ — s- {/ 
such that 

(i) $o is the identity on U, and 

(ii) $ t+s = $t o $ s for any t, s > 0. 

To every continuous vector field F : M m — » R m with unique integral curves is 
associated a semiflow on K m by the equation 

^U t (z) = F($ t (z)), Vx€R m ,Vt eR>„. 
Fix a semiflow $onC/c IR m . 

Definition 3.2 (Invariant set). A set T C f/ is called invariant if $t(r) C T for 
every i > 0. 

Definition 3.3 (Equilibrium point). A point x G U is called an equilibrium if 
$t(x) = x for all t > 0. The equilibrium set of $ is the set of all equilibrium points. 

When <f> is induced by a vector field F, the equilibrium set coincides with the 
set on which F vanishes. 

Definition 3.4 (Chain- recurrent point). Given 5, T > 0, a point x € U is called 
(5, T)-recurrent if there are points xo = x, xi, . . . , Xfe-i, Xk = x £ U and real num- 
bers to,ti,..., tfc_i > T such that 

dist (<J> t< (xi),x i+ i) < 6, i = 0,...,k-l. 

x is said to be chain- recurrent if it is (5, T)-recurrent for any 5, T > 0. 

We denote by CR (<f>) the set of chain-recurrent points. 

Definition 3.5 (Limit set). The limit set of a sequence {x„}„>o C U is the set 

L ({x n } n >o) = {x & U : 3nk t 00 such that x nk — > a;} . 

3.2. A limit set theorem. The reason we can characterize the limit set of the 
random process via the chain-recurrent set of the associated semiflow is due to 
Theorem 1.2 of [T] which, to our purposes, is stated as 

Theorem 3.6. Let F : R m — > R m be a continuous vector field with unique integral 
curves, and let {x(n)}„>o be a solution to the recursion 

x(n + 1) - x(n) = 7„ [F(x(n)) + u n ] , 

where { r y n }n>o is a decreasing gain sequenc^ and {u n } n >n C W 11 . Assume that 
(i) {x(n)} n >o is bounded, and 
(ii) for each T > 0, 

lim sup 

n^oo Y{ fe:0 < Tfe _ T „<r} 

where r„ = Yn=o 7t ■ 
1 1xm n -+oo 7„ = and J2„>o Tn = °°- 



fc-1 

E 



= 0. 
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Then L({x(n)} n >o) is contained in the chain-recurrent set of the semiflow induced 
byF. 

Definition 3.7 (Lyapunov function). Consider a semiflow $ on U C M. m . A 
Lyapunov function for $ is a continuous map L : W C U — > K which is strictly 
monotone along any non-constant orbit of &\w- H W = U, we call L a global 
Lyapunov function and $ a gradient-like system. 

If a gradient-like system $ is generated by a vector field F, we also say that F 
is gradient-like. Under these assumptions, Corollary 3.3 of [l] states that 



Corollary 3.8. Assume the conditions of Theorem \3.6\ and that F is gradient-like 
and has isolated equilibria. Then {x(n)} n >o converges toward an equilibrium. 

First, note that {7 n } ra >o 



3.3. The random process (3.4) satisfies Theorem 

satisfies 



3.6 



Km 7„ = and ^ Jn = 



n>0 



Of course, {x(n)} n >o is bounded. It remains to check condition (ii). For that, let 



i=0 



Observe that {M„}„>o is a martingale adapted to the filtration {J r n } n >o: 

n n 
E [M n+ i \F n ] = ^ li U i + E [ln+lUn+l \F n ] = ^ = M n- 

i=0 i=Q 

Furthermore, because for any n > 

n n 



i=0 



i=0 



the sequence {Af„}„> converges almost surely to a finite random variable (see e.g. 
Theorem 5.4.9 of [3]). In particular, it is a Cauchy sequence and so condition (ii) 
holds almost surely. 

Now, in view of Theorem \3.6\ we will investigate the chain-recurrent set of the 
semiflow generated by the ODE 



( dvi(t) 
dt 



dv m (t) 



dt 



(3.5) 



Given c > 0, let 



A c = {a; € A : Xi + Xj > c,V{z, j} e E) 



On A c , the vector field F of (3.5) is continuous (even Lipschitz). Moreover, we 
have 

Lemma 3.9. For any c < 1/N, A c is invariant under the semiflow induced by 
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Proof. Given € E. 

d 



dt 



(Vi + Vj) 



> 



1 V- «f 

-Vi H > v 

N ^ vf + v? 

k 

-{v% + Vj) 



N ^ vf + v? 



-(Vi + Vj) + 



N 



If v belongs to the boundary of A c , there exists some {i, j} £ E such that vi +vj 
and then 

1 



c, 



■^(Vi+vj) > -(v i + v j ) + ^ 



which means that the vector field F points inward on the boundary of A c . □ 

and Theorem |1.2| will follow from the description of 



The proofs of Theorem 



the chain-recurrent set of (3.5) 



1.1 



4. The case a = 1 for regular graphs: proof of Theorem 11.11 
Here, a = 1 and G is a r-regular, finite, connected graph. According to Theorem 



3.6 the limit set of {x(n)} n >o is contained in the chain-recurrent set CR($) of 
the semiflow $. Therefore it is enough to characterize CR($). We do this by 
introducing a global Lyapunov function for <i>. Let L : A — > K be given by 

m m 

L(vi, . ..,v m ) =logJJui = ^logui. (4.1) 

i=l i=l 

Lemma 4.1. L : A — > K is a global Lyapunov function for the semiflow induced 
by Q . 

Proof. Let v = {vi(t), . . . , v m (tj), t > 0, be an orbit of $. Then 



dt 



(Low) 



d_ 
dt 

m 

E 



E lo s w * 

\i=l 

1 (iwj 



^vA ''■ ' .V E ,. ; . ,. 

i=i y j~i J y 

1 - 1 

-™ + ^7 / / 

iV ^— ' ^ «i + u,- 

i=l J 



We claim that the last expression is nonnegative. For this, note that the above 
summand has 2N terms and, by the arithmetic-harmonic mean inequality, 



„. i 



i=l 



^ + Vi 



EE^+^) 



> (2A0 Z 



(4.2) 
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with equality if and only if 

Vi + vj = const. , V{i, j} G E. 
Since G is r-regular, we have N = rm/2 and 



(4.3) 



+ Vj) = 2r. 



So (4.2) gives that 



EE 



i 



> 



(2N) 2 



2r 



Nm, 



(4.4) 

r: - - r : zr 

2 — 1 J~2 

thus establishing that -^(L o v) > 0. It remains to prove that equality only holds 
along constant orbits. By (4.3), we must have Vi + Vj — const. = r/N for every 
{i,j} G E. Thus 

dVi 1 v i 

= —1): A 

dt 



3~i 



1 

-Vi + — 



\ ' 



N ^ r/N 







and so v is an equilibrium point. 

The proof of Theorem |1.1| is divided into two cases. 



□ 



4.1. Case 1: G is non-bipartite. By the existence of a global Lyapunov func- 
tion, every orbit {v(t)}t>o converges to an equilibrium point. We claim that 
u = (1/m, . . . ,X/m) is the only equilibrium of (3.5). For this, note that every 
equilibrium satisfies 

|(Lo,) = 0. 



By the proof of Lemma 4.1 this happens if and only if (4.3) is satisfied. Fix the 
vertex 1 of G, and let v\ — p and Vi = q for every neighbor i ~ 1. By (4.3), the 
values of Vi along any path in the graph G alternate between p and q, i.e. 

p if the distance from i to 1 is even, 
q if the distance from i to 1 is odd. 

Because G is connected and non-bipartite, it has a cycle of odd length, and so 
p = q. This implies that v — u. It is easy to check that u is indeed an equilibrium 
point of (3.5 ). 

We are thus in the conditions of Corollary 
This proves the first part of Theorem |1.1| 



and so {x(n)} n >o converges to 



4.2. Case 2: G is bipartite. The situation here is quite different from the previ- 
ous one. Let V = AUB be a bipartition of G. Because G is a regular graph, A and 
B have the same cardinality. Observe that, by the same argument of the previous 
section, if v — (vi, . . . , v m ) is an equilibrium point, there exist p, q > such that 

i if i G A, 
/ ifiGB. 



(4.5) 



Because v G A, we have p 
contained in the set 

A 



<l 



2/m. Thus every equilibrium point of (3.5) is 
{v G A : v satisfies (|4.5|)} . 
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Observe that every v G A is indeed an equilibrium point: for i G A 



Fi(v) 



-y 



-p- 



rp 



N(p + q) 



0, 



and the same holds for i € fl. Thus A is the equilibrium set of (3.5 1. 

The goal is to prove that CR ($) = A. This is not guaranteed by the fact that 
each orbit converges toward an element of A. One could have, for example, a 
homoclinic connection: an orbit {v(t)} t >o is born and dies in points of A. In this 
case, the whole orbit {v(t)} t >o is contained in CR($). To avoid such situations, 
we want to prove that A is an attractor. 

Definition 4.2 (Attractor). T is an attractor for $ if 

(i) r is a compact, nonempty, invariant set for <£>, and 

(ii) there exists a neighborhood W of T such that dist ($ t (x),r) — > as t — > oo, 
uniformly in x G W. 

This is not quite true, but almost. What happens is that elements of A are 
attracting equilibria points, but as they approach the two "points at infinity" of A 
(i.e. when p or q goes to zero), some of the eigenvalues of JF on complementary 
directions of A converge to zero, which means that they lose hyperbolicity. We 
bypass this problem by extending the semiflow to neighborhoods of these "points 
at infinity" and showing that the closure A is an attractor for the extended semiflow. 

We first prove that every v G A is an attracting equilibrium, by looking at the 
jacobian matrix 



JF 



dF 1 
dvi 

dF 
dvi 



dFx 

dv m 

dF 

dv m 



(4.6) 



of the vector field F = (Fx, . . . ,F m ) defined by (3.5). Because v G A belongs to a 



line of singularities, is an eigenvalue of JF(v). To be attracting, we must show 
that 

Lemma 4.3. Let v G A. Any eigenvalue of JF(v) different from has negative 
real part, and is a simple eigenvalue of JF(v). 



Proof. We explicitly calculate the entries dFi / dvk ■ Let v G A with Vi 
and Vi — q for i £ B. We have five cases: 



: p for i G A 



• i = k G A: 



m 

dvi 



r s 1 V i 



i = k G B: analogously, 



i ~ k and i G A: 



dFi mp 
dvi (v) = - 1 + ^ 



OF, 



-1 + 



mq 



did 
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• i ~ k and i £ B: analogously, 



(«) 



mq 
~2r' 



i 7^ k: in this case, dFi/dv^ = 0. 



Thus, if we label the vertices of A from 1 to m/2, the vertices of B from 1 to m/2, 
and let M = (m^-) be the m/2 x m/2 adjacency matrix of the edges connecting 
vertices of A to vertices of B (i.e. m»j = 1 when the i-th vertex of A is adjacent to 
the j-th vertex of B), then JF(v) takes the form 

rql —pM 



JF(v) 



rn 
2r 



-qM l rpl 

Letting /i = p/(p + q) and v = q/(p + q), JF(v) can be written as 

rvl —jiM 



JF(v) = -I 



1 



-vM % 



rfil 



=: -I 



1 



S. 



(4.7) 



Given a matrix X, let <j(X) denote its spectrum. By (4.7), 

a(JF(v)) = -a(S) - 1 
r 

and so it is enough to estimate the set a(S). The lemma will follow once we prove 
that 

(a) every element of a(S) is either real or has real part equal to r/2, 

(b) r is the largest real eigenvalue of S, and 

(c) r is a simple eigenvalue of S. 

Let's prove (a). Let A = a + bi € cr(S). Because r^i^rv < r, we can assume that 
A 7^ r/i, rv. Note that the matrix 

(rv - \)I -fiM 

S-XI = 

-vM l (r^i - X)I 

is singular if and only if its Schur complement 



(r/i — \)(rv — A) 

[IV 



MM 1 



(rv - \)1 - (-iiM)(rn - A)- 1 /(-z/M t ) = -t^— 

r/i — A 

is singular. Because MM* is symmetric, all of its eigenvalues are real and so 
(rfj, — A) (rv — A) 



fJLV 



(rfi - \)(rv - A) € 



Calculating the imaginary part of (rfi — X)(rv — A), it follows that 
-rb + 2ab = =*> b = or a = r/2, 

which proves (a). 

To prove (b), let A € a(S) (1 R, say Sx = Ax, where x = (x 1} . . . , x m ) e M m \{0}. 
Letting Xi — max{|xi|, . . . , \x m \} for i £ A, we have 



Xxi 



rvxi — fi 



Xj < rvxi + fi Xi = 
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and thus A < r. The same holds if i € B. 

It remains to prove (c). When A = r, we have for i € A that 



f 1 2-*/ x i 



and similarly for i e B. Thus the function h : V — > K defined by 

Xi if i G A, 



h(i) = 



—Xi if i € B 



is harmonic in G. By the maximum principle, h is constant, and so r is a simple 
eigenvalue of S. □ 

Let {w-oo, Voo} — A\A C 9 A be the points at infinity of A, that is 







if i e A, 
if i€B 



and 



± if i € A. 

o if ieB. 

Observe that, because v_oo,i + v~oo,j,Voo,i + Voo,j > whenever i ~ j, there are 
neighborhoods E/_oo C A of i?_ oc and f/oo C A of t>oo on which the vector field 
F can be extended to a vector field F on A U E/_oo U [/oo. Let $ be the semiflow 
induced by F . Observe that v_ OQ and are equilibria of $. But now 



Ji^-oo) = -I- 



1 



rJ 















am* 



o 



has ra/2 eigenvalues equal to 0, and the same happens to Ji ? (w 00 ). In particular, 
the analysis of eigenvalues does not give any information about the local behavior 
of around these points. We overcome this difficulty by introducing new Lyapunov 
functions on neighborhoods of v_oo and Woo- Let L-oo : C^-oo — > K and Loo : Loo — > 
K be defined by 



L_oo(«) = ^ log Vj 



and Loo (v) = 2J l°g u i 

iGA 



Observe that u_oo is a global maximum of L^^: by the concavity of the log 
function, Jensen's inequality gives that for every v £ [/-oo 

2 

m 



m 

< ¥ log 



L— oo(l'— oo)- 



Analogously, i>oo is a global maximum of L^. 
Lemma 4.4. L 

_oo o,nd LryQ arc Lyapunov functions for the semiflow 

Proof. We only prove the first claim (the second claim is analogous). Like in the 
proof of Lemma |4.1| 



dt 



(L- 



. lit 



N ^ ^ Vi + 

ieB j~i 1 



which, again by the arithmetic-harmonic mean inequality, is nonnegative. Also, it 
is zero if and only if v € A (1 U—oo- 



The existence of the functions L, L 
orbit of $ converges to a point of A. 

We now proceed to establish the following 



□ 



Lqo guarantees, in particular, that every 
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Lemma 4.5. A is an attractor for the semiflow <£>. 
Proof. Given e > 0, let 

X = {x e C/_oo : i-ooCx) > i-oo(«-oo) ~ £o} 

y = {y e ?7oo : ioo(y) > ^00(^00) - £o}- 
These are compact sets containing V-oo and Woo, respectively. Fix x € int(AT) (1 A 



and y € int(Y) D A. They define an interval / of A. By Lemma 4.3 we can take an 



arbitrari ly sm all tub ular neighborhood Z of I that is invariant under Thus, by 



Lemmas 



4.3 



and 



4.4 



the union W = X UY U Z is invariant under <£>. 
A / 







i a ; 




^\ \ V T /' 










I 





Figure 1. The attracting neighborhood X U Y U Z of A. 



It remains to check condition (ii) of Definition 4.2 Observe that because w_oo , 
are global maxima for L^a^L^ respectively and each point of A is uniformly 
attracting, for every e > we can similarly build a smaller invariant set W' — 
X' U Y' U Z' contained in a e-neighborhood of A. Now, since W is compact and 
every point of W converges to a point of A, there is to > such that $>t(W) C W 
for any t > tg. □ 

Now it is an easy task to show that 

Lemma 4.6. CR ($) = A. 

Proof. Obviously, A C CR ($) . For the reverse inclusion, assume by contradiction 
that z A is chain-recurrent. By the previous lemma, we can take a ^-invariant 
compact set W containing A that does not contain z. By construction, W — 3>i(W) 
is also ^-invariant, compact and satisfies 

A c W C int(W). (4.8) 

Let 8 = dist (W', A\int(VF)) > 0. By assumption, there are points zq = z, Z\, . . . , 
Zk-i, z k — z € A and real numbers to,..., t/-_i > 1 such that 

dist(* t4 («i),«i+i) < <5, i = 0,...,A-l. (4.9) 

Thus 3?t (zo) = ( i > tn( z ) e an d so, by (4.9), z\ € W. By induction, we claim 
that z\, Z2, . . ■ ,Zk S W. Indeed, if z, t e W then <&t 4 (zi) € W', and again by (4.9), 
z i+ i 6 W. In particular, 
concludes the proof. 



z G W, which contradicts the choice of W. This 

□ 



A GENERALIZED POLYA'S URN WITH GRAPH BASED INTERACTIONS 



13 



5. The case a < 1 for regular graphs: proof of Theorem 11.21 



We claim that the same L, as defined in (4.1), is a global Lyapunov function 
whenever a < 1. 



Lemma 5.1. For any a < 1, L : A 

semiflow induced by (3.5). 



is a global Lyapunov function for the 



Proof. Let v = (vi(t), . . . ,v m (t)), t > 0, be an orbit of $. Then 
d 



dt 



(Lou) 



d_ 
It 

m 

E 



^logw, 

1 dvi 
Vi dt 



f-f Vi \ Vl + N 4^ vf + v c - 



1 x - V. 



Observe tha£] 



N ^— ' + u" 



a^+lT- 1 2 

> for any x, y > 0, 



x a + y a x + y 
with equality if and only if x = y. Thus 



d 
dt 



/T , 1 U. 

(L o u) = -m + — ~ 



> — m + 



vf+vf ~ " ' N 



1 E 



U< + «7 



(5.1) 



(5.2) 



and this last expression, by inequality (4.4), is nonncgative. 

It remains to check at which points ^(L o v) = 0. This happens if and only if 
equality holds in both (5.2) and (4.4). Equality holds in (4.4) if and only if 

Vi + Vj = const. , V{i, j} € E, (5.3) 



and in (5.2) if and only if 



, V{t,j}EE. 



(5.4) 



Clearly, (5.3) and (5.4) imply that v is the uniform measure. It is easy to check 
that the uniform measure is an equilibrium point. □ 

Thus the semiflow is gradient-like and the uniform measure is the unique equi- 
librium point. By Corollary 3.8 x(n) converges to the uniform measure almost 
surely. 



'The inequality is equivalent to {x a 1 — y a 1 )(x — y) < 0, which is true for a < 1. 
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6. The case of star graphs: proof of Theorem 11.31 
When G is the star graph with to vertices and m is the vertex with higher degree, 



(3.5) becomes 



f dvjjt) 
dt 



~Vi{t) 



1 



Vi{t) a 



m-1 Vi(t) a + v m (t) a 



,m- 1, 



dv m (t) 
dt 



v m (t) a 
v m (t) a +v J {t) a 



-Vm{t) + — T 

to — 1 ' 

i=i 

6.1. Case 1: a = 1. Let L : A — > R be given by 

L(«i, . . .,v m ) = logv 



(6.1) 



(6.2) 



Lemma 6.1. L : A — > K is a global Lyapunov function for the semiflow induced 
by Q. 

Proof. Similar to the proof of Lemma |4.1[ 

d 
dt 

i=i - 

which, by the arithmetic-harmonic mean inequality, is at least 

m — 1 m — 1 to — 1 



m-l 

( iow ) = - 1 + — ,E — 

m — 1 r — ' + V, 



1 



= -1 



1 + (m — 2)1;,, 



> -1 



1 + (m - 2; 



= 0, 



3=1 

with equality if and only if v% = • • • = u OT _i = and w m = 1. It is easy to check 
that (0, . . . , 0, 1) is indeed an equilibrium point of (6.1 ). □ 



By Corollary 3.8 every orbit of $ converges to (0, . . . , 0, 1). 

6.2. Case 2: a > 1. This is the only result we make no use of Lyapunov functions. 
When m = 2, our model is a class of generalized Polya's urn. For simplicity, we 
refer to this process as "g-urn". It is known (see e.g. Theorem 4.1 of [4j) that in 
this case 



lim x(n) — (0, 1) 



> and 



.71— yao 

Now assume m > 2. Observe that, as events 



lim x(n) = (1, 0) 



> 0. 



lim i(n)=fi -!-r,o)}Dn PU^™^) = ^ 

!->oo V TO — 1 TO — 1 / I 111 



i=l n>l 

By a coupling argument, we can identify this last event to the following one: in to— 1 
independent g-urns, just one color of ball is added in each g-urn since the beginning 
of the process. Rubin's Theorem (see e.g. Theorem 3.6 of 12 ) guarantees that 
the event "just one color of ball is added to the g-urn since the beginning of the 
process" has positive probability, and so 

' ! ,0] 0. 



lim x(n) 



TO — 1 



TO — 1 
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To prove the other claim, first observe that 

m— 1 



{Jim x(n) = (0,...,0,l)} D f) f){jj^(n)=0}. 



By a coupling argument, the term on the right hand side of the above inclusion 
has positive probability (again by Rubin's Theorem). This concludes the proof of 
Theorem IQl 



Remark 6.2. Given a finite connected graph G = (V, E), call I C V a,n indepen- 
dent set if {i, j} <^ E for i, j G /. The proof of Theorem 1.3 gives the following: if 
a > 1 and / is an independent set, then 



lim Xi(n) — 0, Vi G / 



> 0. 



7. VARIANTS OF THE MODEL 

7.1. Edges with different weight functions. Let G — (V, E) be a finite, regular, 
connected graph. For each edge {i, j} G E, let fun ■ (0, 1) — > (0, 1). A variant of 
the model is described as follows. Let X\{n — 1), . . . , x m (n — 1) be the proportions 
of balls after step n — 1. At step n, for each edge {i, j} G E add one ball either to 
i or j with probability 

fu j}( x i( n ~ 1)) 

P [i is chosen among {i, j} at step n] = 



f{i,j}{xi(n - 1)) + f^jyixjin - 1)) 



In other words, we replace the law of ^^^(n) in (2.2 1 by the above one, defined in 
terms of the f{ij}'s. 

If we assume that, for each {i, j} G E 1 , the function x i— > f^jj(x)/x is decreasing, 
an inequality similar to ( |5.1[ ) holds: 

f{j,j}(x)/x + f {iJ} (y)/y 2 

— 7 / s , , — / x > for any x, y > 0, 

+ f{i,j}{y) x + y 

with equality if and only if x = y. This implies that the same function L defined in 



(4.1 ) is a Lyapunov function for this variant model. Following the same line of ideas 



in the proof of Theorem 1.2 we conclude that x(n) converges to the uniform measure 
almost surely. A special family of functions satisfying the above assumption is 
f{i,j}( x ) = where a = a({i,j}) < 1. 

7.2. Hypergraph based interactions. We can similarly define a variant of the 
model on hypergraphs. Let G — (V,E) be an hypergraph, where V = {1, . . . ,m} 
and E C 2 V , \E\ = N. Let xi(n — 1), . . . ,x m (n — 1) be the proportions of balls 
after step n — 1. At step n, for each hyperedge e G E add one ball to one of its 
vertices with probability 

_ r . . . xAn — 1) 
P [z is chosen on hyperedge e at step n\ = =^ • 

2jxj-(7l-l) 
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The associated ODE is 

dvi{t) 
dt 



dv m (t) 
dt 



«l(t) 



(7.1) 



.(*)■ 



1 \ ■* v m 



iV 



If G is 

(i) regular: each i g V belongs to a same number of hyperedges, and 

(ii) fc-uniform: \e\ — k for every ee£, 

then L as in (4.1 1 is a Lyapunov function for (7.1 ), by the same arguments used in 



the proof of Lemma 4.1 Thus every orbit {v(t)}t>o converges toward an equilibrium 
point. An equilibrium point v = (v\, . . . , v m ) must satisfy the system of equations 



const. , Ve £ £. 



k}. When k < m, 



(7.2) 



(1/m, . . . , 1/m) 



As a special case, let E — {e e 2 V : |i 
is the only solution to the system of equations (7.2 1. Thus x(n) converges to the 
uniform measure almost surely. When k = m, this variant is a Polya's urn model 
with balls of m colors. See for instance §4.2 of 12 . 



In addition to the above two variants of the model, one can also consider a 
variant in which the number of balls added at each step is some process possibly 
depending on the outcome so far. 



8. Further questions 

This work is part of a program to answer the following 

Problem 8.1. Given a > and a finite connected graph G, what is the limiting 
behavior of x{n) ? 

Now turn attention to the cases we considered. We summarize them in the table 
below. 



G 


Regular non-bipartite 


Regular bipartite 


Star graph 


a < 1 


uniform measure 


uniform measure 


? 


a = 1 


A 


(0,...,0,1) 


a > 1 


? 


? 


Probability > to 
(0, ...,0, 1) and 

(-^ ^- o) 

\m—l ' ' ' ' ' m—l ' J 



Table 1. The limiting behavior of x(n). 



When G is regular bipartite and a = 1, Theorem |l.l| says that L({x(n)} n > ) C A. 
However, we do not know if the limit exists. When the number of vertices is two, 
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the model is the classical Polya's urn, and in this case a well known fact is that 
{x(n)} n >o converges to a point of A almost surely. See e.g. §2.1 of [12]. 



Problem 8.2. For a general regular bipartite graph and a — I, does {x(n)} n >o 
converge to a point of A almost surely? 

Problem 8.3. In Theorems and 1.2 what is the rate of convergence of x(n) 
to its limit? 

This problem is related to the control of the eigenvalues of JF at the singularities 
of F, and of quantitative estimates on the precision that {x(n)}„>o shadows a real 
orbit of See e.g. §3.2 of 12 . For regular non-bipartite graphs and a = 1, direct 
calculations show that every eigenvalue of JF at the uniform measure has negative 
real part. 

Problem 8.4. Let G be a finite connected graph, and let a > 1. What is the 
asymptotic behavior of x(n)? 

This question remains widely open, even when G is a triangle. The uniform 
measure is always a singularity of <1>. When 1 < a < 4/3, it is attracting and 
then x(n) converges to it with positive probability. Also, by Remark 6.2 for any 
i € {1,2,3}, Xi{n) converges to zero with positive probability. So in general, we 
think that there exists a = &o(G) such that when a > ag 



lim x(n) £ dA 



= 1. 



Problem 8.5. Let G be a finite star graph with at least three vertices, and let 
a > 1. What is the asymptotic behavior ofx(n)? 



Item (b) of Theorem 1.3 gives a partial answer. We think that x{n) almost surely 
converges to either (0, . . . , 0, 1) or (l/(m — 1), . . . , l/(m — 1), 0). 

Problem 8.6. Let G be a finite star graph with at least three vertices, and let 
a < 1. What is the asymptotic behavior of x(n)? 

When a — 0, the probability of adding a ball to each extreme of an edge is 1/2. 
By the law of large numbers, 

n^oo v ' \2(m-l)' '2(m-l)'2y 



When a = 1, item (a) of Theorem 1.3 says x(n) converges to (0 
surely. For a < 1, we think that x(n) almost surely converges to 

/ 1 1 



m — 1 + (m — 1) !-<* m — 1 + (to — 1) *-<» to — 1 + (to — 1) 




A direct calculation shows that the above point is the unique equilibrium of (3.5) 
inside A. 

Another question of interest is the following 

Problem 8.7. What is the correlation between the number of balls in the bins, as 
a function of a and of the number of steps n, e.g. when G is an Euclidean lattice? 
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